Repository navigation
chore(test): mutation-test the relay seam and the upload queue - #65
Merged
Merged
Conversation
Merging this PR will not alter performance1 warning
Comparing 
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
mutmut showed that no test pinned what _normalize_relay_path strips. A mutant that stripped whitespace or letters, or that broke the root path, still passed. These tests check that only trailing slashes go, and that near-miss suffixes stay rejected. Signed-off-by: Simon van Lierde <90462640+simonvanlierde@users.noreply.github.com>
…queue - I add mutmut 3.8.0 to the test group and a [tool.mutmut] table. - I add a `just mutation` recipe. It stages a copy with r9cam_models under src/, so mutmut names modules the way the tests import them. - Survivors go to reports/mutation/survivors.txt, and I commit the accepted ones as tests/mutation-baseline.txt. - CONTRIBUTING.md lists the new recipe. Signed-off-by: Simon van Lierde <90462640+simonvanlierde@users.noreply.github.com>
Signed-off-by: Simon van Lierde <90462640+simonvanlierde@users.noreply.github.com>
…mpare them Signed-off-by: Simon van Lierde <90462640+simonvanlierde@users.noreply.github.com>
… tests Signed-off-by: Simon van Lierde <90462640+simonvanlierde@users.noreply.github.com>
- fix the Hypothesis seed under mutmut, which also turns off its example database - list only survived and untested mutants; a timeout means the tests caught it, and timeouts come and go under load - have just mutation diff survivor counts per function against the baseline and fail on a change - re-record the baseline without the two timeout mutants - move mutmut to the dev group so the CodSpeed job does not install it - remove .mutation/ and reports/ in just clean Signed-off-by: Simon van Lierde <90462640+simonvanlierde@users.noreply.github.com>
simonvanlierde
force-pushed
the
mutation-test-relay-queue
branch
from
October 24, 2026 23:05
cc29d19 to
87e5488
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds mutation testing for r9cam_models, app/relay and app/delivery. Nothing graded the tests for the relay trust boundary, the one place where untrusted relay frames enter the app, and the first run left 3 surviving mutants in
_normalize_relay_path: one that stripped whitespace, one that stripped trailing letters, and one that broke the root path.testgroup, with a[tool.mutmut]table that mutates onlyapp/delivery,app/relayandr9cam_models.just mutationruns it. mutmut builds module names from file paths, so the recipe stages a copy with the package undersrc/and runs mutmut there. It starts no database. Survivors go toreports/mutation/survivors.txt, one name per line in the same form as the baseline, sodiffagainst it works._normalize_relay_pathstrips: only trailing slashes,/stays/, and/camera,/cameraXand/camera/Xare still rejected.relay_command_is_allowedand_is_valid_image_idhad no survivors.tests/mutation-baseline.txtholds the 476 that are left (466 survived, 10 with no tests), recorded after the queue property tests from fix(queue): keep the upload queue under its byte cap and add property tests #61. Two equivalent mutants carry a reason; the rest are real gaps, accepted for now. mutmut numbers mutants by position in their function, so the header says to compare counts per function after an edit. There is no CI gate.Tests: the new tests fail against each of the three surviving mutants applied by hand (6, 4 and 2 failures) and pass on the current code.
just mutationexits 0 with 1416 mutants, 940 killed and 466 survived, none in the three target functions, and its survivor list matches the baseline exactly.just checkpasses (866 passed, 1 skipped, 95.0% coverage).